Using Classical Planners for Plan Verification and Counterexample Generation

In this paper, we develop techniques for using classical planning systems to provide decision support to human planners: specifically to help human planners identify and fix problems in their plans that can arise from uncontrolled events in the execution environment. Most automated planning systems assume that the objective of the system is to generate a plan that will achieve some goal or goals in the state of the world. In many practical applications, however, the primary planning responsibility rests with human planners, either because planning systems are not capable of handling the full complexity of the planning application, or because the human users are not willing to cede planning responsibility. In this paper, we show how to use any automated planning system to analyze a plan to identify ways that uncontrolled (disturbance) actions could cause the plan to fail in execution, and produce counterexample traces that would show how failures could occur. The intent is that human planners could use these traces as guidance in improving their plans, perhaps by incorporating our technique in an interactive plan critiquing system. Our proposal is a planning analog to the use of model-checking systems to verify critical hardware and software systems. We describe how our system, MURPHY, translates a plan into what we call a “counter planning” problem, combining a representation of the initial plan with the definition of a set of uncontrolled actions. These uncontrolled actions may be the actions of other agents in the environment, either friendly, indifferent or hostile, or they may be events that simply occur. The result of this translation is a disjunctive planning problem, which we further process in order to play into the strengths of existing classical planners. Using this formulation, a classical planner can find counterexamples that illustrate ways a plan may go awry. We present empirical results in order to demonstrate the practicality of the idea of using classical planners as plan verifiers. Our experiments probe the difficulty of the counter planning problem. We vary the difficulty of the counter planning problems along a number of dimensions, including the number of agents available to the counter-planner, and the agents’ initial configuration. We show how these affect the difficulty of counter-planning. We also compare the use of a classical planner with the use of a more conventional verification tool: NuSMV (Cimatti et al. 2002b). Our results show that even for very difficult planning problems, MURPHY can efficiently generate counter planning planning instances. We also show that for most of the counter planning problems, a PDDL planner (FD (Helmert 2006)) can rapidly compute counterexamples, where they exist, or verify that no counterexample exists. They also show that the planner can do this more efficiently than the general verification system, NuSMV.